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Delivery of mental health care relies upon professionals with 
the latest evidence upon which to base their care. This research 
reports on a pre-test/post-test evaluation of a four-week educa- 
tion program delivered to Pacific Island participants (n = 18) to 
enhance knowledge, skills, and attitudes (KSAs). The education 
program used a combination of formal lectures, tutorials, clinical 
visits, simulations, and laboratory sessions. The measure used was 
the Nurse Self Report (NSR) questionnaire. Results indicate an 
education intervention can be an effective tool for improving the 
knowledge, skills, and attitudes of Pacific Island people who care 
for persons experiencing mental health problems. 



Despite ongoing calls to improve mental health services 
across the globe (Prince, Patel, Saxena et al., 2007; World Bank, 
1993; World Health Organisation [WHO], 2001; WHO Atlas, 
2005), progress towards meeting this challenge has been slow 
in middle- and low-income countries (Saraceno, van Ommeren, 
Batniji, Cohen, Gureje, Maloney, Sridhar & Underbill, 2007). 
Barriers to the improvement of mental health service delivery 
include the existing public health priorities, resistance to de- 
centralisation of services, challenges to the delivery of mental 
health services in primary health care settings, low numbers 
of trained professionals, and the lack of effective leadership 
(Saraceno et al., 2007). The last two challenges identified are 
exacerbated in middle- and low-income countries by the lack 
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of both adequate trairung facilities and opportunities for ongo- 
ing professional development (Saxena, Thomicroft, Knapp, & 
Whiteford, 2007). 

Mental health nurses, generalist nurses, and community 
health workers are fundamental frontline personnel who provide 
mental health services in the Pacific for vulnerable populations 
residing in rural and remote areas. Nurses and/or community 
health workers are often the only available mental health staff. 
Thus, the need for improved training and ongoing professional 
development opportunities is critical. The delivery of mental 
health care in the Pacific relies upon the human resources of pro- 
fessionals rather than technology (Saxena, Thornicroft, Knapp, 
& Whiteford, 2007). Saxena et al. (2007) argue that the cur- 
rent shortages of trained professionals including mental health 
nurses, psychiatrists, psychologists, and social workers, hinders 
the delivery of mental health care in middle- and low-income 
countries. 

At the 2012 Creating Futures Conference, co-hosted by the 
University of Papua New Guinea (UPNG), Queensland Health, 
the National Department of Health PNG, and the Cairns Insti- 
tute, James Cook University (JCU), the poor state of mental 
health in PNG and across the Pacific was discussed (Hunter, 
2012). The JCU WHO Collaborating Centre for Nursing and 
Midwifery Education and Research Capacity Building were 
asked to assist with the development of programs to upskill 
current staff responsible for the delivery of mental health care 
across the Pacific. As a result, a mental health theory and skills 
program designed for nurses and community health workers 
from the Pacific was conducted. The four-week program pro- 
vided theoretical and practical skills to a cohort of mental health 
workers to prepare them to deliver up-to-date, evidence-based 
care to people with a mental illness and to educate and lead 
others working in the area of mental health in the future. 
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In order to determine the efficacy of the program, a pilot 
study using a pre-test/post-test evaluation was conducted. This 
article presents the results of the evaluation. 

METHODS 

Research Design 

The pilot study used a pre-test/post-test design that compared 
18 workshop participants' responses at two different times. The 
initial test was to determine a baseline of participants' self- 
reported mental health knowledge, skills, and attitudes (KSAs). 
The second test was to examine any changes in participants' 
KSAs as a result of attending the four-week mental health ed- 
ucation program. It was hypothesized that participants' knowl- 
edge, skills, and attitudes would improve after the delivery of 
the education program. 

Study Context 

The study was conducted at James Cook University in North 
Queensland, Australia. The University is small in comparison 
with others in Queensland and is located in a regional setting. 

Participants 

A convenience sample of 18 participants (all participants 
who attended the education program) was recruited into the 
study. The participants were from the following Pacific Island 
countries: Cooklslands; Solomon Islands; Palau; Kiribati; PNG; 
Nuie; and Fiji (see Figure 1). An AusAID Australian Fellowship 




nGURE 1. Map of Countries of Participants. Source: Wildniedia Commons 
under Creative Commons Licence CCO. 



Award provided funds for participants to attend the four-week 
course. 

Procedures 

After ethical clearance by the University's Human Research 
Ethics Committee (H5208), participants attending the four-week 
educational program were invited to participate in the study. 
Prior to distribution of the questionnaires a verbal explanation 
supported by an Information Sheet was disseminated to all pro- 
gram attendees. All attendees at the program voluntarily agreed 
to participate and signed a consent form. In order to avoid the 
possibility of coercion, the invitation to participate and the de- 
livery of the pre-test and post-test measures were given by two 
student interns (EC and AG), who were neither members of the 
research team nor teachers in the education program. The par- 
ticipants were evaluated twice: once at baseline (the first day of 
the education program) and at the completion of the education 
program (the last day). 

Instrument 

The pre-test questionnaire collected demographic and per- 
sonal information from respondents including age, gender, qual- 
ifications, previous work experience in mental health settings, 
and previous personal interaction with a person with a mental ill- 
ness. The questionnaire used to evaluate the participants' KSAs 
in both the pre- and post-test was the Nurses Self Report (NSR) 
questionnaire, as modified by Henderson, Happell, and Martin 
(2007) to measure the impact of theory and clinical placement 
on undergraduate students' KSAs. This instrument was origi- 
nally modified from the tool developed by Gillette et al. (1996) 
to evaluate emergency nurses' perceptions of caring for patients 
with mental health problems. Workshop participants were asked 
to identify how accurately each statement on the NSR reflected 
their current KSAs in working with people experiencing mental 
health problems. This was accomplished by circling the most 
appropriate response on an adapted Likert-type four-category 
response scale (1 = Very true, 2 — Fairly true, 3 — Somewhat 
true, 4 = Not at all true). The NSR has previously been found 
to be a reliable instrument, with a reported Cronbach's alpha of 
0.85 (Henderson et al., 2007). The tool was found to be reliable 
in the current study with a Cronbach's alpha of 0.98. Participants 
took approximately 15 minutes to complete the questionnaire. 
The questionnaire was completed anonymously, and anonymity 
was further preserved by having the interns collect the com- 
pleted questionnaires and place them into a sealed box. 

The Education Program 

The education program was developed following the general 
principles of the World Health Organization Mental Health 
Gap Action Program (WHO, 2010): Communication with 
people seeking care and their carers; assessment; treatment 
and monitoring; mobilizing and providing social support; 
protection of human rights; attention to overall well-being; and 
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TABLE 1 

Summary of Grand Challenges Goals Prioritized by Participants and Incorporated into the Program 

GoalB GoalC GoalD GoalE 

Advance prevention and Improve treatments and Raise awareness of the Build human resource 

implementation of early expand access to care global burden capacity 

interventions 



four of the six goals of Grand Challenges in Global Mental 
Health (Table 1) (Collins, Patel, Joestl, March, Insel, & Daar, 
2011). 

The program was based on the best available evidence re- 
lated to mental health, identified as important for the conduct of 
such programs (Patel, Garrison, de Jesus Mari, Minas, Prince 
& Saxena, 2008; Collins, Patel et al., 2011), and integrated ex- 
isting priorities for mental health care (WHO, 2010; Collins, 
Patel et al., 2011). The program design also took into account 
other factors, including cultural context, English language profi- 
ciency of participants, various education/experience levels, and 
needs identified at the earlier workshop in PNG. These factors 
were similar to those taken into account by the developers of a 
mental health training course delivered in Haiti after the 2010 
earthquake (Cianelli et al., 2013). 

The program was conducted for four weeks, excluding week- 
end days. Lectures and invited presentations were conducted in 
the mornings followed by workshops, tutorials, simulated ex- 
ercises, and laboratory-based activities in the afternoon. Addi- 
tionally, specialist lecturers, clinicians from local mental health 
services, consumers, and consumer advocates delivered sessions 
(see Table 2). 

Data Analysis 

Data generated by the questionnaires were analysed using 
SPSS (Statistical Package for the Social Sciences), version 20 



(IBM SPSS Inc., Chicago, IL, USA). Frequencies, percentages, 
means, modes, and standard deviations were calculated to de- 
termine patterns in the data. Wilcoxon Signed-Rank tests were 
used to identify differences between the baseline pre-test and 
final post-test NSR scores. Spearman's Rank Order correlation 
was performed to determine differences between continuous 
variables. Mann-Whitney tests were used to analyse relation- 
ships between nominal and continuous variables. Results were 
calculated for the sample as a whole then calculated for partic- 
ipants stratified by age, gender, nurse type, and mental health 
qualifications. 



RESULTS 

Demographic Characteristics 

The majority of workshop participants were female (61.1%), 
were aged under 45 years (72.2%), had previously worked in a 
mental health setting as a mental health nurse (77.8%), and all 
(100%) had a previous personal interaction with a person with a 
mental illness. The most common formal nursing or health care 
qualification was a Bachelor degree (33.3%) followed by an Ad- 
vanced Diploma (25%). Other highest qualifications/training in- 
cluded a certificate (8.3%), a post-graduate certificate (16.8%), 
a diploma (8.3%), and a Masters degree (8.3%). Overall, 44.4% 
of participants had specific mental health qualifications as well 
as nursing or health worker qualifications. 



TABLE 2 
Program Summary 



Workshops 



Mental Health Simulation 



VisitsA'^isitors 



• Teaching theories & styles 

• Education pedagogy & philosophy 

• Mental health promotion 

• Leadership & governance 

• Reflective practice 

• Clinical supervision 

• Evidence-based practice 

• Resilience 

• Perinatal mental health 

• Mental health first aid 

• Mental health promotion poster 
presentation 



• Mental health nursing roles 

• Mental health assessment 

• Mental state assessment 

• Psychosocial assessment 

• Mental health disorders 



• Consumer consultant 

• Local mental health nurses 

• Local non-government services 

• James Cook University library 
& computer rooms 

• Private mental health hospital 

• Public mental health unit 

• Aboriginal & Torres Strait 
Islander cultural centres 
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Knowledge, Skills, and Attitudes 

Stage 1: Baseline Data 

The mean for each item on the NSR stage 1 (NSRl) was 
calculated for the study sample and ranked from 1 (indicat- 
ing highest confidence), to 22 (indicating lowest confidence) 
(Table 3). The highest ranked KSAs were: referral of men- 
tal health clients; respectful communication; working with 
those with self-inflicted injuries; and performing assessments of 
psychotic symptoms and mental status. Participants also were 
reasonably confident about their ability to triage a mental health 
problem, differentiate between delirium and dementia, and as- 
sess potential for violence. The lowest ranking KSAs suggest 
that the participants were less confident in their ability to un- 
derstand victims of repeated domestic violence, identify legal 
parameters, and assess suicide risk. Of the 22 KSAs, the partici- 
pants reported being least confident in their ability to understand 
clients who injure themselves or others while driving under the 
influence of substances. 

There was no significant difference in self-reported KSAs 
between males and females nor according to age. Mental health 
nurses reported higher confidence compared with community 
health workers in assessing potential for violence (p — 0.025) 
and assessing psychotic symptoms {p — 0.046), but the aggre- 



Rank 

1 

2 
3 
4 
5 
6 
7 
8 

9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 



1 = Very true, 2 = Fairly true, 3 = Somewhat true, 4 = Not at all true. 
Lower mean scores indicate higher confidence. 



gated KSA scores were not significantly different between the 
two groups. 

Stage 2: Post-Test upon Completion of the Education Program 
The mean score for each NSR stage 2 (NSR2) item was cal- 
culated for the entire sample and ranked from 1 to 22, from 
most to least confidence. These rankings were then compared 
to the NSRl rankings from baseline to determine any changes 
(Table 4). Mean scores on all 22 items on the NSR2 indicated 
higher KSAs for participants after the mental health education 
program, and all scores, apart from the lowest two, indicated 
high confidence (>2.0). The difference between the two ag- 
gregated mean scores (NSRl and NSR2) was not statistically 
significant. 

A key outcome from this study is that after attending the 
mental health education program, mean scores on all items, 
apart from two, reduced to <2, which indicates confidence in 
20 KSA items. These results show an improvement on pre-test 
scores in which only three items scored <2. 

The KSAs ranked highest in the NSRl were consistent with 
the KSAs ranked highest in the NSR2: Referral of mental health 
clients, performing assessments of psychotic symptoms, triag- 
ing mental health problems, and respectful communication. 



TABLE 3 

NSRl: Knowledge, Skills, and Attitudes Ranked from Highest to Lowest Confidence at Baseline 



Mean NSRl Questions 



1.77 I am able to refer mental health clients to appropriate services. 

1.88 My conununication with mental health clients is respectful. 

1 .94 I always know when to refer a client for psychiatric consultation. 

2.05 I work well with patients with self-inflicted injuries. 

2. 11 I am able to communicate effectively with people who are suspicious or paranoid. 

2.16 I am skilful in assessing psychotic symptoms. 

2.16 I am skilful in performing a mental status assessment. 

2.16 It is as easy for me to triage a mental health problem as it is to triage a physical 
health problem. 

2.22 I am skilful in differentiating between delirium and dementia. 

2.22 I am skilful in assessing potential for violence. 

2.22 I am able to detect side-effects of psychoactive drugs. 

2.22 My communication with clients with self-inflicted injuries is respectful. 

2.23 My communication with clients affected by alcohol is respectful. 
2.25 I utilize psychiatric terminology appropriately. 

2.27 My communication skills enable me to calm anxious people. 

2.29 I find it easy to understand clients who harm themselves. 

2.33 I am skilful in differentiating substance intoxication and psychosis. 

2.33 I understand the therapeutic actions of psychoactive drugs. 

2.38 I find it easy to understand clients who are victims of repeated domestic violence. 

2.50 I am able to identify the legal parameters that relate to mental health clients. 

2.50 I am skilful in assessing suicide risk. 

2.70 I find it easy to understand clients who injure themselves or others while driving 
under the influence of substances. 



AN EDUCATIONAL PROGRAM FOR MENTAL HEALTH NURSES 



341 



TABLE 4 

NSR2: Knowledge, Skills, and Attitudes Ranked from Highest to Least Confidence from Baseline to Post Program 



NSR2 


NSR2 




NSRl 


NSR 1 


Rank 


Mean 


NSR2 Questions 


Rank 


Mean 


1 


1.39 


1 am able to refer mental health clients to appropriate services. 


1 


1.77 


2 


1.50 


1 always know when to refer a client for psychiatric 
consultation. 


3 


1.94 


3 


1.50 


1 am skilful in assessing psychotic symptoms. 


6 


2.16 


4 


1.50 


It is as easy for me to triage a mental health problem as it is to 
triage a physical health problem. 


8 


2.16 


5 


1.55 


My communication with mental health clients is respectful. 


2 


1.88 


6 


1.61 


1 am skilful in performing a mental status assessment. 


7 


2.16 


7 


1.61 


My communication with clients affected by alcohol is 
respectful. 


13 


2.23 


8 


1.66 


I find it easy to understand clients who are victims of repeated 
domestic violence. 


19 


2.38 


9 


1.66 


I am able to communicate effectively with people who are 
suspicious or paranoid. 


5 


2.11 


10 


1.66 


My communication with clients with self-inflicted injuries is 
respectful. 


12 


2.22 


11 


1.66 


My communication skills enable me to calm anxious people. 


15 


2.27 


12 


1.77 


I am able to detect side-effects of psychoactive drugs. 


11 


2.22 


13 


1.77 


I work well with patients with self-inflicted injuries. 


4 


2.05 


14 


1.77 


I understand the therapeutic actions of psychoactive drugs. 


18 


2.33 


15 


1.83 


I utilize psychiatric terminology appropriately. 


14 


2.25 


16 


1.83 


I am skilful in differentiating substance intoxication and 
psychosis. 


17 


2.33 


1 n 
1 / 


1 QC 
1 .oo 


I am skilful in assessing potential for violence. 


1 f\ 


L.LL 


18 


2.00 


I find it easy to understand clients who injure themselves or 
others while driving under the influence of substances. 


22 


2.70 


19 


2.00 


I find it easy to understand clients who harm themselves. 


16 


2.29 


20 


2.00 


I am able to identify the legal parameters that relate to mental 
health clients. 


20 


2.50 


21 


2.05 


I am skilful in assessing suicide risk. 


21 


2.50 


22 


2.06 


I am skilful in differentiating between delirium and dementia. 


9 


2.22 



1 = Very true, 2 = Fairly true, 3 = Somewhat true, 4 = Not at all true. 
Lower mean scores indicate higher confidence. 

Participants reported a large increase in confidence in respect- tions) were compared. Figure 2 shows all three aggregated 
fully communicating with clients affected by alcohol and under- scores reduced indicating higher confidence in KSAs post-test 
standing clients who are victims of repeated domestic violence, compared with pre-test. The difference in pre- and post-test 
Participants also were reasonably confident about their commu- scores for Knowledge (p = 0.048), Skills (p = 0.021), and 
nication skills with mental health clients in situations as detailed Attitudes (p = 0.047) was significant, 
in Table 4. The lowest three ranking KSAs did not change greatly 
apart from participants' confidence in their ability to differen- 
tiate between delirium and dementia, which reduced from a DISCUSSION 

ranking of 9 to the lowest ranking item of 22. This research was undertaken to measure the efficacy of a 
There was no difference in self-reported KSAs between mental health education program designed specifically for par- 
males and females, mental health nurses and community health ticipants from Pacific island countries with the aim to improve 
workers, according to age, or those with and without mental knowledge, skills and attitudes. The program was based on the 
health qualifications. best available evidence, developed following the general princi- 
Pre-test and post-test aggregated Knowledge scores (5 ques- pies of the World Health Organization Mental Health Gap Ac- 
tions), Skills scores (9 questions), and Attitudes scores (8 ques- tion Program (WHO, 2010) and four of the six goals of Grand 
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25 




5 



Pre-test Post-test 
—♦—Knowledge 10.93 8.27 
-■-Skills 20.27 16.33 
Attitudes 18.06 13.88 



FIGURE 2. Changes in Knowledge, Skills, and Attitudes: Pre- and Post-Test 
Mean Scores. 

Challenges in Global Mental Health (Collins, Patel et al., 201 1), 
and using culturally and contextually appropriate information to 
ensure its specificity and applicability for the participant group. 
In addition, the educational intervention has been designed to 
incorporate a variety of pedagogical techniques to enhance the 
teaching and learning experience for mental health workers from 
Pacific islands, and it incorporated practical sessions with local 
clinicians, consumers, and carers to increase social awareness 
and related issues. This is similar to the findings of Cianelli 
et al. (2013), who also found that a short, specifically designed 
mental health training program has the ability to improve nurses 
and other health workers' mental health knowledge, attitudes, 
and skills. However, in this project, the educational intervention 
differs from those that focus primarily on content delivery. The 
inclusion of mental health consumers and carers in the education 
program, alongside visits to local clinical facilities, proved to be 
an effective strategy to enhance change in participant attitudes. 

This study is also unique as it measures the influence of 
an education intervention on the KSAs of nurses and commu- 
nity health workers from Pacific island countries who care for 
persons experiencing mental health issues. There is a scarcity 
of research that reports the professional development needs of 
Pacific island nurses. Given the lack of specialized profession- 
als in the area of mental health and the recent World Health 
Organization (2013) call for an increase in, and more equi- 
table distribution of, mental health professionals in low- and 
middle-income countries, the findings of this research provides 
important support for the need for ongoing professional devel- 
opment for mental health workers in Pacific island countries, as 
suggested by Saxena et al. (2007). 

Many of the participants of the education program are lead- 
ers or hold important management positions in their respective 
countries. The improvement in the knowledge of the participants 
in this study may assist to improve the provision of mental health 
care and leadership in represented Pacific island countries, given 
people who hold positions of power within ministries of health 



and departments of public health often have low levels of essen- 
tial mental health knowledge (Chisholm et al., 2007). 

Limitations 

While the research indicates promising results, it has a num- 
ber of limitations. First, the small sample size means the results 
are not generalizable. A larger sample size would allow for ex- 
amination of the relationships between variables, especially the 
effect of the training program for participants from different 
countries. As this was a pilot test of a newly developed edu- 
cation intervention program, a larger sample will be available 
after the course is rolled out on a larger scale. 

Second, the use of the convenience sample from one delivery 
of the education program also limits the generalizability of the 
findings. However, as a pilot study of the first delivery of the 
education program, the results are extremely valuable. 

Third, the questionnaire was validated by Henderson, Hap- 
pell, and Martin (2007) and amended by the authors (KU and 
TP), but not tested on the cohort prior to the study. Therefore, 
the reliability and validity of the tool with this cohort is currently 
unclear. 

Fourth, the study was limited by self-report data and by 
the social desirability inherent in all self-report measures. Self- 
report questionnaires may contain bias because individuals tend 
to endorse socially desirable knowledge, skills, and attitudes 
and under-report socially undesirable knowledge, skills, and 
attitudes. However, the pre-test/post-test study design is more 
effective than a single post-test program evaluation because each 
participant serves as his or her own control (Nimon, Zigarmi & 
Allen, 2011). 

Implications for Practice and Research 

This research indicates that the education program developed 
and delivered to the 18 participants from Pacific island countries 
was beneficial. Participants reported the education program had 
a meaningful impact on their knowledge, skills, and attitudes for 
the delivery of mental health care. This research extends pre- 
vious research that reports effective outcomes from education- 
based training programs. This research is seminal to the field as 
there is nothing currently available to inform the development 
of educational interventions for mental health workers in the 
Pacific islands. Research should be undertaken after the next 
delivery of the education program to further evaluate the effec- 
tiveness of the program, explore the need for new content to be 
introduced into the program, and explore the outcomes of the 
program in participants' countries of origin. 

Declaration of interest: The authors report no conflicts of 
interest. The authors alone are responsible for the content and 
writing of the paper. 
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